Second-Order Guarantees of Stochastic Gradient Descent in Nonconvex Optimization

Recent years have seen increased interest in performance guarantees of gradient descent algorithms for nonconvex optimization. A number works uncovered that noise plays a critical role the ability recursions to efficiently escape saddle-points and reach second-order stationary points. Most available limit component be bounded with probability one or sub-Gaussian leverage concentration inequalities arrive at high-probability results. We present an alternate approach, relying primarily on mean-square arguments show more relaxed relative bound variance is sufficient ensure efficient from saddle points without need inject additional noise, employ alternating step sizes, rely global dispersive assumption, as long direction every point.